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(landmarks) whose positions are known in a three-dimensional array are arranged in the 
real space. Then, the external parameter of the camera is corrected using an error between 
the actual position of a target landmark, included in some landmarks captured by the 
camera and displayed on the display screen of the camera, and the position of the target 
landmark predicted based on the position and orientation of the camera at the time, 
acquired by a position and orientation sensor,H M ^^*'^^ M " M ^"' 



Please replace the paragraph starting at page 2, line 23, and ending at page 3, 
line 4, with the following replacement paragraph. A marked-up copy of the paragraph 
starting at page 2, line 23, and ending at page 3, line 4, showing the changes made thereto, 
is attached. 

another method, when there are a plurality of landmarks in the real space, 



using three (or one or two) landmarks in the image captured by a camera, and a 
measurement value measured by a position and orientation sensor, a projective matrix from 
J the real space (three-dimensional space) to the display screen (two-dimensional plane) of 
the camera is calculated by a matrix operation, and an external parameter of the camera is 
corrected using the obtained matrixV - 



IN THE CLAIMS: 



Please cancel Claim 5 without prejudice to or disclaimer of the recited subject 



matter. 
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Please amend Claims 1,3, and 6-21 as follows. Please add new Claim 24 as 
follows. A marked-up version showing the amendments to the claims is included in the 
attached appendix. For the Examiner's convenience, all of the pending claims are 
presented, regardless of whether the claims is currently being amended. 



1 . (Amended) A position and orientation determination apparatus^mch 
identifies a parameter indicating a position and orientation of a capture unit adapted to 
capture a picture in a real space containing a plurality of feature paims whose positions are 
known in a three-dimensional array, comprising: 

a position and orientation measurement umfadapted to measure the position 
and orientation of the capture unit in a method otirer than using a captured picture; 

a detection unit adapted to deteotihe plurality of feature points and their 
positions in a two-dimensional array on an image pickup screen using the picture in the 
real space captured by the capture unit; 

a prediction um^adapted to predict the positions of the feature points in the 
two-dimensional arra^n the image pickup screen based on the position and orientation of 
the capture unit measured by said position and orientation measurement unit; and 

correction unit adapted to correct a parameter of the measured position and 
orientation of the capture unit based on the positions of the feature points on the image 
pioJcup screen of the capture unit obtained by said prediction unit, and based on the 
positions of the feature points obtained by said detection unit 
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2. (Unamended) The^e^ttTon and orientation determination apparatus 
according to claim 1, whg^n an artificially applied marker is used as the plurality of 
feature points whogethree-dimensional positions are known. 



3. (Amended) Th^osition and orientation determination apparatus according 
to claim 1 , wherein a pointtoriginally existing in the real space as the plurality of feature 
points whose three-dirarensional positions are known, and whose two-dimensional positions 
can be detected onine image pickup screen by said detection unit. 



4. (Amended) The position and orientation determination apparatus according 
to claim 1, further comprising: 

an image j^jtfbvgiion unit adapted to generate an image of a virtual object based 
on the corrected parameter, and to superimpose the image of the virtual object on the 
picture in the real space captured by the capture unit. 



Claim 5 has been cancelled. 




^ (Amende^^he position and orientation determination apparatus according 
^^^to claim 1, whereii^^d position and orientation measurement unit is a sensor for 
measuring the potion and orientation of the capture unit. 
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f) p*. (Amended) The position and orientation determination apparatus according 
to claimjl' wherein said sensor is a magnetic sensor for measuring the three-dimensionaj 
position and orientation of the capture unit. 



sT (Amended) The position and orientation determination apparatus according 
to claim 1, wherein said correction unit corrects the parameter such that t^e capture unit 
either rotates or translates. 



v.. 



% (Amended) The position and orientation deternpnation apparatus according 
to claim 1, wherein said correction unit corrects the parameter of the capture unit by 
combining rotation transform with translation transform 



). (Amended) The position and orientation determination apparatus 
according to claims, wherein said correction/unit corrects the parameter of the capture unit 
by combining rotation transform with translation transform alternately and plural times. 

V 

Yl . (Amended) The position and orientation determination apparatus 

i 

according to claim# wherein sap correction unit corrects the parameter such that the 
capture unit can rotate, and thfn re-predicts the two-dimensional position of the feature 
point on the image pickup^creen based on the position of the feature point in the real space 
and the position and orientation of the camera after the correction, and the capture unit can 
translate. 
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XI. (Amended) The position and orientation determination apparatus 
according to claim -9fwherein said correction unit corrects the parameter such thajrthe 
capture unit can translate, and then re-predicts the two-dimensional position o^the feature 
point on the image pickup screen based on the position of the feature poinffin the real space 
and the position and orientation of the camera after the correction, and^ne capture unit can 
rotate. 

«. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said correction unit computes a first average value on the 
image pickup screen of the capture unit using the position of the feature point obtained by 
said prediction unit, and a second average value onraie image pickup screen of the capture 
unit of the feature point detected by said detection unit, and corrects the parameter such 
that the first average value matches the sec^d average value. 

yf. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said/orrection unit computes a first average weighting value 
on the image pickup screen of im capture unit using the position of the feature point 
obtained by said prediction unit, and a second average weighting value on the image 
pickup screen of the captup unit, of the feature point detected by said detection unit, and 
corrects the parameter s^uch that the first average weighting value matches the second 
average weighting value. 
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>5. (Amended) The position and orientation determination apparatus 
according to claim J^, wherein when the parameter of the capture unit is processedjplural 
times alternately by rotation transform and translation transform, said correctionionit 
repeats the process until an average value or an average weighting value of anierror 
between the position of the feature point corrected by said prediction unit awd the position 
of the feature point on the image pickup screen is equal to or less than a predetermined 
value or until the error cannot be smaller. 

fix 

JS- (Amended) The position and orientation determination apparatus 



W IS 

whe 



according to claim l^wherein when the parameter of the caprare unit is processed plural 
times alternately by rotation transform and translation transform, said correction unit 
repeats the process until an average value or an average fighting value of an error 
between the position of the feature point corrected by^aid prediction unit and the position 
of the feature point on the image pickup screen is e^fual to or less than a predetermined 
value or until the error cannot be smaller. 



47. (Amended) The position dfiA orientation determination apparatus 
according to claim lj^ wherein when sajfi correction unit corrects the parameter such that 
the capture unit can rotate, the featurjfpoint whose position predicted by said prediction 
unit is associated with the feature doint detected by said detection unit, a rotation axis and a 
rotation angle with which the position of the feature point on the image pickup screen of 
the capture unit using the posrtion of the feature point obtained by said prediction unit 
matches the position of theffeature point on the image pickup screen of the capture unit 



V 



obtained by said detection unit are obtained for each feature point, and the parameter can 
be corrected by using average values of the rotation axis and the rotation angle obtained^- 
each feature point which is dealt with. 



' Jti (A 



8. (Amended) The position and orientation determination apparatus 

ip is 

according to claim Xfi wherein when said correction unit corrects the parameter such that 
the capture unit can rotate, the feature point whose position predictedJvy said prediction 
unit is associated with the feature point detected by said detection Jtnit, a rotation axis and a 
rotation angle with which the position of the feature point on the image pickup screen of 
the capture unit using the position of the feature point obtaipd by said prediction unit 
matches the position of the feature point on the image pickup screen of the capture unit 
obtained by said detection unit are obtained for eaclyfeature point which is dealt with, and 
the parameter can be corrected using average weighting values of the rotation axis and the 
rotation angle obtained for each feature pointwhich is dealt with. 



/ 
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(Amended) The position and orientation determination apparatus 
l^l^fw 



according to claim^o, wherein when said correction unit corrects the parameter such that 
the capture unit can translate, therfeature point whose position predicted by said prediction 
unit is associated with the feamre point detected by said detection unit, a difference 
between the position of the feature point obtained by said prediction unit and the position 
of the feature point objlained by said detection unit is obtained for each feature point which 
is dealt with, and thS parameter can be corrected using average values of the difference 
obtained for eaon feature point which is dealt with. 
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(Amended) The position and orientation determination appanj#f^^ 
according to claim wherein when said correction unit corrects th^arameter such that 
the capture unit can translate, the feature point whose positio^redicted by said prediction 
unit is associated with the feature point detected bysara detection unit, a difference 
between the position of the feature point obt^ffed by said prediction unit and the position 
of the feature point obtained by said defection unit is obtained for each feature point which 
is dealt with, and the parametercan be corrected using average weighting values of the 
difference obtained foj^ach feature point which is dealt with. 

21. (Amended) A position and orientation determination apparatus which 
identifies a parameter indicating a position and orientation of a capture unit ad^ted^G 
capture a picture in a real space containing a plurality of feature points whose>positions are 
known in a three-dimensional array, comprising: 

a position and orientation measurement unit adantea to measure the position 




and orientation of the capture unit inya/method other than using a captured picture; 

a detection unit adapte^^etect^ie^lurality of feature points and their 
positions in a two-dimensional array on aplmage pickup screen using the picture in the 



real space captured by the capture urflt; 

lit adapted 1 



on^ap'i 



a prediction unit adapted to predict the positions of the feature points in the 
two-dimensional arra^em the image pickup screen based on the position and orientation of 
the capture unit fffeasured by said position and orientation measurement unit; and 

'a correction unit adapted to correct a parameter of the measured position and 
orientation of the capture unit based on the positions of the feature points on the image 
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pickup screen of the cap^^fnit obtained by said prediction unit, and based on the 
positions of the feature points obtained by said detection unit. 



22. (Unamended^The program code which performs the position and 
orientation determinm^method according to claim 21. 

23. (Unamended) The storage medium storing the program code according to 



Please add new Claim 24 as follows: 



V~\ ^^-JP*- (New) An information processing methodxomprising the steps of: 



acquiring a measurement resu]^>btained by a sensor which measures a position 
and orientation ofj 

detecting a plurality of landmarks from an image captured by the capture unit, 
generating a vieWng transform condition based on the measurement result, 
generating a^orrection condition using the plurality of landmarks detected in 
said detecting step, 

genera/ling a virtual image based on the viewing transform condition and the 
correction condition, and 

combining the generated virtual image and the captured image.— 
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